Parallel Computing is a compelling vision of how computation can seamlessly scale from a single processor to virtually limitless computing power. Unfortunately, the scaling of application performance has not matched peak speed, and the programming burden for these machines remains heavy. The applications must be programmed to exploit parallelism in the most efficient way possible. Today, the responsibility for achieving the vision of scalable parallelism remains in the hands of the application developer.

This book represents the collected knowledge and experience of over 60 leading parallel computing researchers. They offer students, scientists and engineers a complete sourcebook with solid coverage of parallel computing hardware, programming considerations, algorithms, software and enabling technologies, as well as several parallel application case studies. The Sourcebook of Parallel Computing offers extensive tutorials and detailed documentation of the advanced strategies produced by research over the last two decades
application case studies. The Sourcebook of Parallel Computing offers extensive tutorials and detailed documentation of the advanced strategies produced by research over the last two decades
* Provides a solid background in parallel computing technologies
* Examines the technologies available and teaches students and practitioners how to select and apply them
* Presents case studies in a range of application areas including Chemistry, Image Processing, Data Mining, Ocean Modeling and Earthquake Simulation
* Considers the future development of parallel computing technologies and the kinds of applications they will support

A guide to advanced features of MPI, reflecting the latest version of the MPI standard, that takes an example-driven, tutorial approach. This book offers a practical guide to the advanced features of the MPI (Message-Passing Interface) standard library for writing programs for parallel computers. It covers new features added in MPI-3, the latest version of the MPI standard, and updates from MPI-2. Like its companion volume, Using MPI, the book takes an informal, example-driven, tutorial approach. The material in each chapter is organized according to the complexity of the programs used as examples, starting with the simplest example and moving to more complex ones. Using Advanced MPI covers major changes in MPI-3, including changes to remote memory access and one-sided communication that simplify semantics and enable better performance on modern hardware; new features such as nonblocking and neighborhood collectives for greater scalability on large systems; and minor updates to parallel I/O and dynamic processes. It also covers support for hybrid shared-memory/message-passing programming; MPI_Message, which aids in certain types of multithreaded programming; features that handle very large data; an interface that allows the programmer and the developer to access performance data; and a new binding of MPI to Fortran.

Foreword by Bjarne Stroustrup Software is generally acknowledged to be the single greatest obstacle preventing mainstream adoption of massively-parallel computing. While sequential applications are routinely ported to platforms ranging from PCs to mainframes, most parallel programs only ever run on one type of machine. One reason for this is that most parallel programming systems have failed to insulate their users from the architectures of the machines on which they have run. Those that have been platform-independent have usually also had poor performance.Many researchers now believe that object-oriented languages may offer a solution. By hiding the architecture-specific constructs required for high performance inside platform-independent abstractions, parallel object-oriented programming systems may be able to combine the speed of massively-parallel computing with the comfort of sequential programming.Parallel Programming Using C++ describes fifteen parallel programming systems based on C++, the most popular object-oriented language of today. These systems cover the whole spectrum of parallel programming paradigms, from data parallelism through dataflow and distributed shared memory to message-passing control parallelism.For the parallel programming community, a common parallel application is discussed in each chapter, as part of the description of the system itself. By comparing the implementations of the polygon overlay problem in each system, the reader can get a better sense of their expressiveness and functionality for a common problem. For the systems community, the chapters contain a discussion of the implementation of the various compilers and runtime systems. In addition to discussing the performance of polygon overlay, several of the contributors also discuss the performance of other, more substantial, applications.For the research community, the contributors discuss the motivations for and philosophy of their systems. As well, many of the chapters include critiques that complete the research arc by pointing out possible future research directions. Finally, for the object-oriented community, there are many examples of how encapsulation, inheritance, and polymorphism can be used to control the complexity of developing, debugging, and tuning parallel software.

The thoroughly updated edition of a guide to parallel programming with MPI, reflecting the latest specifications, with many detailed examples. This book offers a thoroughly updated guide to the MPI (Message-Passing Interface) standard library for writing programs for parallel computers. Since the publication of the previous edition of Using MPI, parallel computing has become mainstream. Today, applications run on computers with millions of processors; multiple processors sharing memory and multicore processors with multiple hardware threads per core are common. The MPI-3 Forum recently brought the MPI standard up to date with respect to developments in hardware capabilities, core language evolution, the needs of applications, and experience gained over the years by vendors, implementers, and users. This third edition of Using MPI reflects these changes in both text and example code. The book takes an informal, tutorial approach, introducing each concept through easy-to-understand examples, including actual code in C and Fortran. Topics include using MPI in simple programs, virtual topologies, MPI datatypes, parallel libraries, and a comparison of MPI with sockets. For the third edition, example code has been brought up to date; applications have been updated; and references reflect the recent attention MPI has received in the literature. A companion volume, Using Advanced MPI, covers more advanced topics, including hybrid programming and coping with large data.

Message Passing Interface (MPI) ist ein Protokoll, das parallel Berechnungen auf verteilten, heterogenen, lose-gekoppelten Computersystemen ermoglicht."

This book presents an easy-to-read discussion of domain decomposition algorithms, their implementation and analysis. The relationship between domain decomposition and multigrid methods is carefully explained at an elementary level, and discussions of the implementation of domain decomposition methods on massively parallel super computers are also included. All algorithms are fully described and explained, and a mathematical framework for the analysis and complete understanding of the methods is also carefully developed. In addition, numerous numerical examples are included to demonstrate the behaviour of this important class of numerical methods. This book is ideal for graduate students about to embark on a career in computational science. It will also be a valuable resource for all those interested in parallel computing and numerical computational methods.

The major research results from the Scalable Input/Output Initiative, exploring software and algorithmic solutions to the I/O imbalance.As we enter the "decade of data," the disparity between the vast amount of data storage capacity (measurable in terabytes and petabytes) and the bandwidth available for accessing it has created an input/output bottleneck that is proving to be a major constraint on the effective use of scientific data for research. Scalable Input/Output is a summary of the major research results of the Scalable I/O Initiative, launched by Paul Messina, then Director of the Center for Advanced Computing Research at the California Institute of Technology, to explore software and algorithmic solutions to the I/O imbalance. The contributors explore techniques for I/O optimization, including: I/O characterization to understand application and system I/O patterns; system checkpointing strategies; collective I/O and parallel database support for scientific applications; parallel I/O libraries and strategies for file striping, prefetching, and write behind; compilation strategies for out-of-core data access; scheduling and shared virtual memory alternatives; network support for low-latency data transfer; and parallel I/O application programming interfaces.